Fine-Tuning GPT-3 for Russian Text Summarization

Automatic summarization techniques aim to shorten and generalize information given in the text while preserving its core message most relevant ideas. This task can be approached treated with a variety of methods, however, not many attempts have been made produce solutions specifically for Russian language despite existing localizations state-of-the-art models. In this paper, we showcase ruGPT3 ability summarize texts, fine-tuning it on corpora news their corresponding human-generated summaries. Additionally, employ hyperparameter tuning so that model’s output becomes less random more tied original text. We evaluate resulting texts set metrics, showing our solution surpass performance without additional changes architecture or loss function. Despite being able sensible summaries, model still suffers from number flaws, namely, is prone altering Named Entities present (such as surnames, places, dates, etc.), deviating facts stated document, repeating summary. #COMESYSO1120.